Examination of the Samviq Methodology for the Subjective Assessment of Multimedia Quality

The Video Quality Experts Group (VQEG) is preparing a large number of co-ordinated subjective multimedia quality tests. These tests will be performed by laboratories located in Europe, Asia and North America. The subjective data will be used to evaluate the performance of competing objective multimedia quality assessment algorithms. The reliability of the subjective test data is therefore of great importance for VQEG’s task. Although standards were produced on the basis of best available evidence obtained from related fields, relatively few multimedia subjective quality studies were performed to address specifically the most appropriate and reliable assessment method for multimedia. There is a need for more substantive research evidence to be published in order to validate and improve existing multimedia quality assessment standards. ITU-T SG9, ITU-R WP6Q and VQEG are supporting activities on developing new methods for assessing the subjective quality of multimedia systems. Recently, a new multimedia subjective quality assessment method, named SAMVIQ, has been proposed for standardisation within ITU-R WP6Q. SAMVIQ was originally an EBU methodology and has been designed specifically for assessing the perceptual quality of video delivered as part of a multimedia service. In this paper, two psychovisual experiments investigating the suitability of SAMVIQ are presented. The results of these experiments are discussed within the context of the VQEG multimedia testing programme.